1. Field of the Invention
This invention relates generally to the processing of the compressed domain representation of image data, and more particularly to the manipulation of the compressed domain representation to achieve certain spatial domain processing, such as regular geometric transformations of an image, without having to subject the image data to the full decompression and compression process.
2. Description of the Related Art
A typical high quality digitized color image may use 24 bits per pixel (bpp)--8 bits each for red (R), green (G) and blue (B) in RGB color space or for luminance (Y), chrominance (C.sub.B) and chrominance (C.sub.R) in YC.sub.B C.sub.R color space. To transmit or store such images in the uncompressed state (i.e., in spatial or pixel domain) is simply too costly in terms of time and memory requirements. Thus, applications and devices which store and/or transmit high quality digitized color images, such as digital cameras, typically do so in a compressed format, using one of the currently available compression algorithms.
The emergence of compression standards such as JPEG (an acronym for "Joint Photographic Experts Group") has led to many digital imaging systems and applications that create and maintain content only in JPEG compressed format. For instance, in most digital still-imaging cameras (DSCs) such as the Epson PhotoPC 600, Kodak DC-10, etc., pictures captured by the camera are immediately compressed within the camera and stored in the camera's storage system as JPEG files. Often there is a need to manipulate these pictures prior to display. Typical image manipulations might include (a) rotating the picture from portrait to landscape mode and vice-versa, (b) scaling the picture to increase or decrease its size, (c) changing brightness and contrast in the picture, (d) cropping portions of the picture for the purposes of creating a new picture and for compositing operations, (e) adding simple bitmap annotations to a picture, and (f) embedding visible/invisible watermarks in the picture. Due to storage constraints within the digital camera, these image manipulations require the processed output to be in JPEG format.
The need to do these tasks and the availability of the picture only in the compressed mode has resulted in a great deal of interest in developing image processing techniques that can be applied directly to the compressed domain representation. The motivation for investigating compressed domain processing methods stems from the observations that (a) the volume of data in compressed domain tends to be quite small compared to the spatial domain representation which means that fewer operations per sample may be required for the desired image processing task, and (b) conventional processing pipelines that require the data to be decompressed, followed by the application of the desired image processing function in spatial domain, and then recompressed for transmission or storage efficiency can lead to a loss in image fidelity. Furthermore, such a conventional processing pipeline has very high computation complexity or high latency since the compression task is often more complex than the decompression task. The compressed-domain based processing methodology on the other hand, often leads to reduced computation complexity since it replaces the JPEG decompression and compression tasks by low complexity tasks such as Huffman decoding and Huffman encoding. (See, S. F. Chang and D. G. Messerschmitt, "Manipulation and Compositing of MC-DCT Compressed Video," IEEE JSAC Special Issue on Intelligent Sigital Processing, vol. 13, no. 1, pp. 1-11, January 1995; N. Merhav and V. Bhaskaran, "A fast algorithm for DCT-domain inverse motion compensation," Proc. ICASSP '96, pp. IV.2307-2310, Atlanta, May 1996; B. Natarajan and V. Bhaskaran, "A fast approximate algorithm for scaling down digital images in the DCT domain," IEEE International Conference on Image Processing (ICIP), Washington, D.C., October 1995; and Brian Smith and Larry Rowe, "Algorithms for manipulating compressed images," IEEE Computer Graphics and Applications, pp. 34-42, September 1993.)